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INTRODUCTION 


Reference  is  made  to  the  war  of  words  between  R.  Haralick  on  one  hand  and  W. 
Cirimsonand  E.  Hildreth  on  the  other.1,2’1  The  discussion  centered  around  determining  step 
edges  from  noisy  digital  pictures  wherein  Haralick  purported  to  prove  that  theMarredge 
detector  did  not  do  as  well  as  the  Prewitt  operator  and  neither  did  as  well  as  the  Directional 
Derivative  edge  operator  developed  by  Haralick.  Grimson  and  Hildreth  point  out  that 
Haralick  made  improper  use  of  the  Marr  operator,  an  observation  with  which  we  agree. 

The  Marr  operator  was  developed  as  a  model  for  mammalian  visual  systems  where  a 
raw  primal  sketch  of  an  image  is  developed  numerically  by  calculating  edge  pictures  at 
several  resolutions.  The  full  primal  sketch  is  constructed  by  grouping  the  raw  primitives  in  a 
variety  of  ways.4  A  brief  description  of  the  Marr  operator  and  our  approach  to  computing 
the  edge  picture  is  given  in  appendix  A. 

Haralick  suggests  that  the  digital  picture  should  be  regarded  as  a  representation  of  the 
true  image  where  random  noise  has  been  added  to  each  pixel.  We  suggest  that  this  is  a  far  too 
simple  representation  of  the  imaging  event.  The  real  difficulty  stems  from  clutter,  where 
sought-for  detail  is  obscured  by  other  real  world  detail.  Since  no  edge  finder  can  locate 
hidden  edges  without  a  knowledge  base  to  fill  in  the  blanks,  we  are  more  concerned  at  this 
time  in  using  edge  finders  in  an  interactive  mode  to  detect  real  world  lines  and  edges.  One  of 
the  first  developments  along  these  lines  was  demonstrated  at  ETL  on  the  DI  AL  system.  In 
that  experiment,  a  drainage  pattern  was  extracted  interactively  from  a  digital  image.5 
Essentially,  we  operated  in  a  computer-aided  mode  where  the  Marr  operator  extracted  the 
drainage  pattern  along  with  a  host  of  unwanted,  yet  real  world,  edges.  Image  processing 
functions,  which  are  described  in  appendix  C.  were  used  interactively  to  extract  the  drainage 
pattern  from  the  unwanted  detail. 

The  purpose  of  the  work  described  in  this  Research  Note  is  to  demonstrate  that  the 
Marr  edge  finder  compares  favorably  to  the  Prewitt  and  to  Haralicks's.  especially  when 
simple  image  processing  functions  are  applied  to  the  edge  picture.  The  images  used  in  this 
work  were  similar  to  the  noisy  checkerboards  used  by  both  authors.  Since  most  edge  finders 
w  ork  well  when  the  edge  is  apparent,  reasonably  well  w  hen  only  faintly  obscured  and  not  at 
all  when  the  edge  is  totally  obscurred.  we  feel  that  the  argument  as  to  the  best  algorithm  may 
be  similar  to  a  tempest  in  a  teapot. 

’  K.  I  f  .1 1  <i  liu  k .  “Digital  Step  f  el  ill's  hoiii  /eto  i'rossmg  ot  Second  I  )irectiona  I  I  >c  r  i  \  ati\ es."  Iff!  ftansai  n<>n\  Pattern 
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NUMERICAL  EXPERIMENT 
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Performance  Measures 

Three  criteria  were  used  to  judge  the  performance  of  the  Marr  operator.  The  first 
criterion  (Pj)  is  the  conditional  probability  of  an  assigned  edge  given  that  the  pixel  was  in 
fact  a  true  edge.  The  second  criterion  ( P2)  is  the  conditional  probability  of  a  true  edge  given 
that  the  process  assigned  an  edge  value  to  the  pixel. 

Pj  =  Prob  [Marr  Edge  True  Edge] 

P*>  =  Prob  [True  Edge  Marr  Edge] 

These  creteria  are  identical  to  the  probability  measures  used  by  Haralick  and  Crimson  and 
Hildreth. 
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Haralick  defined  his  error  distance  as  “the  average  distance  to  closest  true  edge  pixel  of 
pixels  which  are  assigned  nonedge  but  which  are  true  edge  pixels."  A  straightforward 
interpretation  of  this  definition  is  that  the  average  error,  and  indeed  any  such  error,  is  zero. 
We  define  the  error  distance  as  the  average  distance  to  the  closest  true  edge  pixel  or  pixels 
which  are  assigned  to  be  edge  pixels. 

E  =  NERNEQ 
NEQ  =  Number  of  Marr  Edges 
NER  =  Sum  of  Marr  Error  Distances 


Checkerboard  #1  -  Noise  Model  #1 

A  number  of  simulation  runs  as  described  in  appendix  B  were  run  on  the  DIAL  system. 
Three  of  the  resulting  noisy  checkerboards  are  described  in  figures  1 . 2.  and  3.  The  five  noisy 
scenes  (CTN  =  10,  20, 30,40, 50)  were  processed  using  four  Marr  resolution  w  indow  s  (°M  =  2, 
3,  4.  5).  The  resulting  edge  pictures  were  further  processed  using  the  raster  cleansing 
algorithms  described  in  appendix  C.  The  performance  criteria  for  the  four  Marr  resolutions 
are  presented  in  Tables  1.  2.  3,  and  4.  Note  that  each  set  of  results  are  presented  in  two 
groups,  namely,  those  results  (# I )  where  thickening  and  thinning  were  applied  and  those 
results  (#2)  where  they  were  not  applied. 


Table  1.  Checkerboard  ffl  -  Noise  Model  SI  -  °M  =  2 

#1  32 

1*2  E  P|  P2 

)0  1.000  0.00  1.000  0.984  0. 


aN  CL  UK  =  LIN 

10  25  1 

20  29  I 


Pi 


■> 


1 


Table  2.  Checkerboard  #1  -  Noise  Model  tt  1  -  °M  =  3 
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Fable  3.  Checkerboard  #1  -  Noise  Model  #1  -  °M  =  4 
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Table  4. 

Checkerboard  tt  1  - 
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In  order  to  get  a  visual  description  of  the  process,  compare  figures  4. 5  and  6  to  figures  I . 
2  and  3.  Figures  4.  5  and  6  are  the  resulting  edge  pictures  when  the  Marr 
resolution  window  (°M  =  5)  is  applied  to  figures  I.  2  and  3,  When  the  raster  processing 
exercise  is  applied  to  figures  4,  5  and  6,  the  results  can  be  seen  in  figures  7.  8  and  9. 

Note  the  amount  of  noise  in  figure  3  when  aN  =  50.  Consider  figure  10.  which  is  the 
resultant  edge  window  derived  from  figure  3,  when  the  Marr  resolution  was  °M  =  2.  This 
figure  demonstrates  that  low  resolution  values  of  the  Marr  operator  act  as  high  pass  filters. 
In  this  case,  the  resultant  edge  image  contained  too  many  noise  edges  for  raster  processing  to 
be  effective.  This  is  the  reason  tables  I  and  2  are  incomplete. 
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Edge  pictures  at  several  resolutions  were  regarded  by  Marr  and  Hildreth  as  a  raw 
primal  sketch  of  the  edge  scene.  A  more  complete  primal  sketch  may  be  obtained  by  various 
groupings  of  the  raw  sketches,  A  sensible  combination  of  the  raw  sketches  should  produce  a 
more  complete  sketch  of  the  edge  scene  and  in  our  case,  a  more  accurate  representation.  In 
order  to  test  this  notion  we  combined  several  of  the  processed  results  in  pairs  by  declaring  an 
edge  at  a  pixel  if  each  member  of  the  pair  also  declared  the  pixel  to  bean  edge.  The  results  are 
presented  in  tables  5,  6  and  7.  The  headings  #  I  and  #2  have  the  same  meaning  as  before. 


Table  5.  (aM  =  5)  •  AND  •  (CTM  =  2) 

#1 


aN 

Pi 

P2 

E 

Pi 

P2 

E 

10 

1.000 

0.984 

0.02 

0.993 

1.000 

0.00 

20 

1.000 

0.983 

0.02 

0.977 

0.999 

0.00 

30 

0.944 

0.965 

0.14 

0.829 

0.978 

0.13 

Table  6. 

(°M  =  5)  •  AND  •  (°M  =  3) 
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Checkerboard  #1  -  Noise  Model  #2 


In  this  example,  the  average  gray  shade  of  the  16  X  16  =  256  squares  remains  exactly  as 
in  the  first  example.  The  standard  deviation  of  the  noise  was  allowed  to  vary  for  each  of  the 
squares  as  described  in  appendix  B.  The  resultant  noisy  checkerboard  is  shown  in  figure  1  1 . 
It  was  processed  using  three  Marr  resolutions  (CTM  =  5, 4  and  3).  The  resulting  edge  pictures 
were  further  processed  using  the  raster  cleansing  algorithms  and  the  results  are  presented  in 
Iable8. 
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Checkerboard  #2 

In  this  example,  the  average  gray  shade  of  the  256  squares  varied  as  did  the  standard 
deviation  of  each  square.  The  random  model  is  described  in  appendix  B.  The  resultant  noisy 
checkerboard  is  shown  in  figure  1 2.  It  was  processed  using  two  Marr  resolutions  (CTM  =  5  and 
4).  The  resulting  edge  pictures  were  further  processed  using  the  raster  cleansing  algorithms 
and  the  results  are  presented  in  table  9. 

Table  9.  Checkerboard  #2 

°M 

CL 

LIK  =  LTN 

Pi 

P2 

E 

4 

29 

0 

0.379 

0.250 

7.88 

5 

29 

0 

0.317 

0.265 

7.37 

5 

29 

2 

0.342 

0.243 

7.60 

Discussion 

The  primary  reason  for  this  work  was  to  show  that  the  Marr  operator,  when  properly 
used,  compares  favorably  with  the  Prewitt  and  Haralick  operators.  This  does  not  mean  we 
endorse  the  Marr  operator  over  all  others.  In  fact,  it  seems  at  this  time  that  more  effort 
should  be  expended  by  researchers  toward  developing  knowledge  bases  for  filling  in 
obscured  edges  than  tweaking  the  multitude  of  edge  operators,  most  of  which  are 
comparable  in  performance. 

Haralick  reported  that  the  Marr  operator  is  in  some  cases  biased  and  that  it  and  other 
edge  operators  must  produce  edges  2  pixels  wide.  The  latter  statement  depends  on  how  the 
edge  is  defined.  We  defined  an  edge  in  such  a  way  that  it  occurred  at  specific  pixels.  Consider 
figure  13  below,  which  depicts  a  portion  of  a  checkerboard  where  the  squares  are  4  by  4. 
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Haralick  defines  an  edge  pixel  as  one  which  has  a  neighbor  that  has  a  different  value  than  the 
pixel;  the  two  pixel-wide  corridors  defined  by  the  dashed  lines  are  edge  pixels  according  to 
Haralick.  We  defined  a  pixel  to  be  an  edge  pixel  if  its  neighbor  to  the  right  had  a  different 
value,  and  or  if  its  neighbor  just  below  it  had  a  different  value.  Thus,  the  pixel  columns  and 
pixel  rows  indicated  by  arrows  are  edges  according  to  our  definition.  When  we  processed  a 
noiseless  checkerboard  with  the  Marr  operator,  we  found  P|  =  P2  =  1. 000  and  E  =  0.00, 
which  indicates  an  unbiased  processor. 

An  examination  of  tables  1  through  4  indicates  that  the  line-thickening  line-thinning 
process  had  a  slight  positive  effect  on  the  error  E  and  a  slight  negative  effect  on  the 
conditional  probability  estimate  P2.  The  operation  had  a  significant  positive  effect  on  P|. 
That  the  Marr  operator  compares  favorably  with  the  Prewitt  and  Haralick  operators  can  be 
checked  by  noting  that  P 1  =  0.674  and  P2  =  0.687  for  the  Prewitt  operator  and  that  P 1  =  0.72 1 
and  P2  =  0.720  for  the  Haralick  operator.  From  table  4,  P |  =  0.7 1 2  and  P2  =  0.705  for  the 
Marr  operator  when  line-thickening,  line-thinning  was  not  used.  From  the  same  table,  P[  = 
0.814  and  P2  =  0.709  for  the  Marr  operator  when  line-thickening,  line-thinning  was  used. 
Haralick  determined  that  P|  =  0.721  and  P2  =  0.716  for  the  Marr  operator  when  he  used  a  50 
by  50  window  rather  than  an  1 1  by  1 1  as  he  did  originally.  Note  that  the  conditional 
probabilities  in  this  paragraph  pertain  to  the  experiment  where  °N  =  50  (Noise)  and  °M  =  5 
(Marr  Resolution). 

Consider  tables  5,  6  and  7  and  especially  table  5.  The  results  displayed  in  these  tables 
demonstrate  a  marked  improvement  in  the  error  E  when  members  of  the  raw  primal  sketch 
are  combined  as  described  in  the  section  on  Numerical  Experiments.  The  improvement  in  E 
improves  most  when  pairs  from  the  raw  primal  sketch  set  are  combined  that  differ  most  in 
their  resolution  values.  The  same  is  true  for  the  conditional  probabilities  P|  and  P2  Note 
that  P2  in  the  combined  form  improves  over  the  P2  components  from  the  raw  primal 
sketches  whereas  the  conditional  probability  P|  diminishes  somewhat. 
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The  only  difference  between  (Checkerboard  #1  -  Noise  Model  42)  and  (Checkerboard 
#1  -  Noise  Model  #1 )  is  that  the  noise  standard  deviation  varied  from  square  to  square  in  the 
former  and  was  fixed  at  five  levels  (aN  =  10,  20,  30,  40.  and  50)  for  the  latter.  The  noise 
standard  deviation  varied  randomly  between  aN  =  0  and  °N  =  50  for  (Checkerboard  #1 
-Noise  Model  #2)  as  described  inappendix  B.  The  results  for  this  modelshould  besimilarto 
the  average  results  of  (Checkerboard  #1  -  Noise  Model  #1).  The  validity  of  this  supposition  is 
supported  when  table  8  is  compared  to  tables  2,  3,  and  4. 

Checkerboard  42,  described  in  appendix  B.  should  and  did  cause  more  problems  than 
any  of  the  other  models  considered  in  this  work.  The  gray  shade  average  varied  randomly 
(75  <GS  <  1 75)  from  square  to  square  as  did  the  standard  deviation  of  the  noise(0  <  aN  < 
50).  The  resultant  noisy  image  is  shown  in  figure  1 2  and  the  processed  results  are  shown  in 
table  9.  T  his  model  may  be  more  realistic  than  the  other  models  considered  in  this  report. 
The  likelihood  of  adjacent  squares  (fields?)  having  nearly  the  same  reflectance  values  is  not 
remote  in  reality  nor  in  this  model  Should  both  squares  have  large  and  nearly  equal  noise 
properties,  then  the  boundary  might  very  well  be  impossible  to  discern. 

I  he  purpose  of  the  work  described  in  this  report  was  to  demonstrate  that  the  Marredge 
finder  compares  favorably  to  the  Prewitt  and  Haralick  edge  finders  especially  when  simple 
raster  processing  functions  are  applied  to  the  Marredge  picture.  The  numerical  results  given 
in  the  third  paragraph  of  this  section  show  this,  at  least  as  far  as  noisy  checkerboards  are 
concerned  It  appears  that  the  performance  differences  among  the  several  edge  finders  is 
trivial  when  consideration  is  given  to  the  fact  of  their  inability  to  find  obscured  edges.  Until 
considerable  research  is  performed  in  the  development  of  knowledge  bases  to  help  locate 
hidden  edges  we  teel  that  the  current  crop  of  edge  finders  would  be  best  used  in  an 
inie rat 1 1 v e  mode  wheie  anv  supporting  image  processing  functions  such  as  those  described 
m  appendix  <  sould  be  used  to  enhance  the  sought-for  edges. 


(  <»(  I. UNIONS 


1  I  in  Mail  edite  tinder  does  not  appear  to  be  a  biased  operator. 

I  he  Mart  edge  tinder  compares  tavorabh  in  performance  with  the  Prewitt  and 
Haram  k  edge  findcis 

*  I  sing  noisv  chcckct hoards  in  an  edge  detector  comparative  analysis  produces  results 
ah. nit  noisv  checkerboards,  a  phenomenon  tew  people  find  useful. 

4  (  lutter  not  random  noise,  causes  most  of  the  problems  for  edge  finders. 

5  future  comparative  analvses  should  use  cluttered  aerial  images  as  bases  for  tests. 

6  Knowledge  bases  need  to  be  developed  to  fill  in  obscured  edges. 


APPENDIX  A 


Marr  Edge  Operator 

The  Marr  edge  detector  is  defined  to  be  the  La  Placian  of  the  Bivariate  Normal  density 
function.  Symbolically,  the  operator  is  represented  as  A2  G(x,y)  where  A2  is  the  La  Placian 
and  G(x,y)  is  the  normal  density  function. 

G(x,y)  =  (1/2  tt  a1)  e  -  1/2  a-  (x2  +  y2) 

A2:  Second  derivative  operator. 

A2G(x,y)  =  (2a2  -  R2)  e 

R2  =  x2  +  y2 

Note  that  the  coefficient  1,  (27ra*)  of  A2G(x,y)  has  been  discarded  and  that  the  function  was 
multiplied  by  -I.  Zero  values  of  Z  =  A2G(xy)*I(xy)  are  determined  where  represents  a 
windowing  operation  and  I(x,y)  represents  the  image. 

The  function  G(x,y)  is  a  smoothing  function  where  large  values  of  a  (resolution 
parameter)  cause  the  operator  to  act  as  a  low  pass  filter.  Zero  values  of  the  second  derivative 
are  those  places  where  the  gray  shade  intensity  change  is  greatest. 

We  approximated  the  Marr  operator  with  a  49  by  49  window  for  all  of  the  resolution 
parameters  used  in  this  report.  Convolution  techniques  were  used  to  develop  Z.  An  edge 
picture  was  developed  by  determining  those  pixel  locations  of  Z  where  a  zero  crossing 
occurred.  We  did  not  threshold  the  zero  crossings  to  determine  an  edge  point.  The  edge 
picture  was  calculated  bv  replacing  a  pixel  w  ith  a  “1”  if  a  zero  crossing  occurred,  otherwise 
the  pixel  was  replaced  with  “0”. 


APPENDIX  B 


Checkerboard  Models 

The  basic  uncontaminated  checkerboard  pattern  used  in  this  work  is  larger  than  the 
pattern  used  by  Haralick  or  by  Grimson  and  Hildreth.  In  their  examples,  they  used  a  200  by 
200  image  where  each  of  the  squares  of  the  checkerboard  was  20  by  20.  We  used  a  512  by  512 
image  where  each  square  was  32  by  32.  In  both  cases,  ours  and  theirs,  the  grayshade  value  of 
dark  squares  was  75  and  that  of  the  light  squares  was  175.  We  considered  two  noise  models 
for  the  basic  checkerboard. 

Noise  Model  #1 

A  random  number  generator  was  used  to  estimate  normal  deviates  N(o,0N)  and  these 
estimates  were  added  to  each  pixel  of  the  uncontaminated  checkerboard.  We  allowed  0N  to 
be  10, 20, 30, 40  and  50.  Haralick  and  Grimson  and  Hildreth  considered  only  0N  =  50.  If  the 
gray  shade  value  plus  noise  was  less  than  zero,  the  resultant  gray  shade  was  set  to  zero.  If  the 
gray  shade  value  plus  noise  was  greater  than  255,  the  resultant  gray  shade  was  set  to  255. 

Noise  Model  #2 

A  random  number  generator  was  used  to  estimate  normal  deviates  N(o,  0IJ)  and  these 
estimates  were  added  to  each  pixel  of  the  uncontaminated  checkerboard.  The  standard 
deviation  for  the  IJth  square  (I  =  1,  16  and  J  =  1,  16)  was  itself  determined  in  a  random 
manner. 


oIJ  =  50  *  R 

where  R  for  each  IJ  was  randomly  selected  from  the  rectangular  distribution 
R('/2,  1/12) 

Neither  Haralick  nor  Grimson  and  Hildreth  considered  this  model. 

Checkerboard  Model  #2 

In  this  model  both  the  IJth  average  gray  shade  value  and  the  IJth  standard  deviation 
were  selected  in  a  random  manner.  The  characteristics  of  the  IJth  square  (I  =  I,  16  and  J  =  1, 
16)  are  defined  as  N  (Gy  ojj). 

Gjj  =  75  +  100  *  R] 

on  =  50  *  R2 

where  R|  and  R2  were  randomly  selected  from  the  rectangular  distribution 
R('/2.  1/12). 

Neither  Haralick  nor  Grimson  and  Hildreth  considered  this  model. 
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APPENDIX  C 


Image  Processing  Functions 

Three  image  functions,  namely  clutter  removal,  line  thickening  and  line  thinning,  were 
used  interactively  to  clean  up  edge  pictures  derived  from  the  Marr  operator. 

Clutter  Removal 

This  function  is  used  to  erase  blobs  or  unconnected  raster  bit  patterns  of  various  sizes. 
In  this  work,  the  symbol  “CL”  defines  the  dimension  of  a  square  window'  which  is  centered 
on  every  pixel  of  the  edge  picture.  If  no  data  passes  through  the  window  or  lies  on  the 
window  boundary,  then  all  interior  data  is  erased. 

Line  Thicken 

This  function  is  used  to  connect  line  breaks  and  to  fill  in  holes.  A  3  by  3  window  is 
placed  over  each  pixel  of  the  edge  picture  where  if  the  center  value  is  “1”  then  the  eight 
neighbors  are  converted  to‘Ts”.  In  this  work,  the  symbol“LIK"  defines  the  number  of  times 
the  basic  line  thickener  was  applied  to  the  edge  picture.  Note  that  the  three  image  processing 
functions  are  in  effect  applied  in  a  parallel  fashion.  This  means  that  the  operators  are  applied 
to  the  original  edge  picture  for  all  pixels  and  a  processed  edge  picture  is  produced. 

Line  Thinner 

This  function  is  used  to  reduce  a  thickened  line  of  arbitrary  pixel  width  to  a  center  line 
of  no  more  than  one  pixel  in  width.  Again  a  3  by  3  window  is  placed  over  each  pixel  of  the 
edge  image  and  the  algorithm  decides  whether  or  not  to  erase  the  center  pixel.  There  are  256 
cases  for  the  algorithm  to  consider.  In  this  work  the  symbol  “LTN”  defines  the  number  of 
times  the  basic  line  thinner  was  applied.  Note  that  the  process  will  not  disconnect  portions  of 
a  connected  thick  line  no  matter  how  many  times  the  process  is  applied.  Thus,  a 
disconnected  line  on  the  original  edge  picture  may  be  thickened  enough  times  to  fill  in  the 
disconnects  and  then  thinned  enough  times  to  produce  a  connected  one  pixel  wide  line. 


